Co-Occurrence Vectors From Corpora Vs. Distance Vectors From Dictionaries

A comparison W~LS made of vectors derived by using ordinary co-occurrence statistics from large text corpora and of vectors derived by measuring the interword distances in dictionary definitions. The precision of word sense disambiguation by using co-occurrence vectors frorn the 1987 Wall Street Journal (20M total words) was higher than that by using distance vectors from the Collins English l)ictionary (60K head words + 1.6M definition words), llowever, other experimen-tal results suggest that distance vectors contain some different semantic information from co-occurrence vectors. 1 I n t r o d u c t i o n Word vectors reflecting word meanings are expected to enable numerical approaches to semantics. Some early attempts at vector representation in I)sycholinguistics were the semantic d(O'erential approach (Osgood et al. 1957) and the associative distribution apl)roach (Deese 1962). llowever, they were derived manually through psychological experiments. An early at tempt at automation was made I)y Wilks el aL (t990) us-. ing co-occurrence statistics. Since then, there haw" been some promising results from using co-occurrence vectors, such as word sense disambiguation (Schiitze [993), and word clustering (Pereira eL al. 1993). llowever, using the co-occurrence statistics requires a huge corpus that covers even most rare words. We recently developed word vectors that are derived from an ordinary dictionary by measuring the interword distances in the word definitions (Niwa and Nitta 1993). ' this method, by its nature, h~s no prol)lom handling rare words. In this paper we examine the nsefldness of these distance vectors as semantic re W resentations by comparing them with co-occur,'ence vectors. 2 D i s t a n c e V e c t o r s A reference network of the words in a dictionary (Fig. 1) is used to measure the distance between words, q'he network is a graph that shows which words are used in the. definition of each word (Nitta 1988). The network shown in Fig. 1 is for a w~ry small portion of the reference network for the Collins English 1)ictionary (1979 edition) in the CI)-I{OM I (Liberman 1991), with 60K head words -b 1.6M definition words. writing unit (Or) \ / word comnmnieation / ~ alphMmtlcal \ L _ \ / l a n g u a g , dictionary o, / \ ( : ) p,.~ople ~,ook (O~) Fig. 1. Portion of a reference network. For example, tile delinition for diclionarg is % book ill which the words of a language are listed alphabetically . . . . " The word dicliona~d is thus linked to the words book, word, language, and alphabelical. A word w~etor is defined its the list of distances from a word to a certain sew of selected words, which we call origins. The words in Fig. 1 marked with Oi (unit, book, and people) m'e assumed to be origin words. In principle, origin words can be freoly chosen. In our exl~eriments we used mi(Idle fi'equency words: the 51st to 1050th most frequent words in the reference Collins English I)ictiotmry (CI';D), The distance w~ctor fl)r diclionary is deriwM it'* fob lOWS: ~ ) ... disti~uc,, ((ticl., 01) dictionary ~ 1 . . . distance (dict., 0'2) 2 . . . distance (dicL, Oa) The i-4,h element is the distance (the length of the shortest path) between diclionary and the i-th origin, Oi. To begin, we assume every link has a constant length o[' 1. The actual definition for link length will be given later. If word A is used in the definition of word B, t.he,m words are expected to be strongly related. This is the basis of our hypothesis that the distances in the refi~rence network reflect the associative distances between words (Nitta 1933).